LSTM-Based Mixture-of-Experts for Knowledge-Aware Dialogues
نویسندگان
چکیده
We introduce an LSTM-based method for dynamically integrating several wordprediction experts to obtain a conditional language model which can be good simultaneously at several subtasks. We illustrate this general approach with an application to dialogue where we integrate a neural chat model, good at conversational aspects, with a neural question-answering model, good at retrieving precise information from a knowledge-base, and show how the integration combines the strengths of the independent components. We hope that this focused contribution will attract attention on the benefits of using such mixtures of experts in NLP. 1
منابع مشابه
Phone-aware Neural Language Identification
Pure acoustic neural models, particularly the LSTM-RNN model, have shown great potential in language identification (LID). However, the phonetic information has been largely overlooked by most of existing neural LID models, although this information has been used in the conventional phonetic LID systems with a great success. We present a phone-aware neural LID architecture, which is a deep LSTM...
متن کاملSpoken Term Detection for Persian News of Islamic Republic of Iran Broadcasting
Islamic Republic of Iran Broadcasting (IRIB) as one of the biggest broadcasting organizations, produces thousands of hours of media content daily. Accordingly, the IRIBchr('39')s archive is one of the richest archives in Iran containing a huge amount of multimedia data. Monitoring this massive volume of data, and brows and retrieval of this archive is one of the key issues for this broadcasting...
متن کاملThe Effect of Role-Play through Dialogues vs. Written Practice on Iranian Intermediate EFL Learners’ Knowledge of English Idioms
This study aimed to investigate the effect of role-play through dialogues vs. written practice on Iranian intermediate EFL learners’ knowledge of English idioms. The question this study tried to answer is if role-play through dialogues vs. written practice has a significant effect on Iranian intermediate EFL learners’ knowledge of English idioms. To find the answer to the question, ...
متن کاملOutrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
The capacity of a neural network to absorb information is limited by its number of parameters. Conditional computation, where parts of the network are active on a per-example basis, has been proposed in theory as a way of dramatically increasing model capacity without a proportional increase in computation. In practice, however, there are significant algorithmic and performance challenges. In t...
متن کاملGrounding New Words on the Physical World in Multi-Domain Human-Robot Dialogues
This paper summarizes our ongoing project on developing an architecture for a robot that can acquire new words and their meanings while engaging in multidomain dialogues. These two functions are crucial in making conversational service robots work in real tasks in the real world. Household robots and office robots need to be able to work in multiple task domains and they also need to engage in ...
متن کامل